Software implementation errors are one of the most significant contributors to information system security vulnerabilities, making software testing an essential part of system assurance. Exhaustive testing—testing all possible combinations of inputs and execution paths—is impossible for real-world software, so high assurance software is tested using methods that require extensive staff time and thus have enormous cost. For less critical software, budget constraints often limit the amount of testing that can be accomplished, increasing the risk of residual errors that lead to system failures and security weaknesses.
Web application Quality Assurance (QA) Engineers or test developers can use automated testing tools to test that the application components are performing as expected. For example, a user, such as a QA Engineer or test developer, can use the automated testing tool to develop test cases for testing components of an application. Although a tool can automate the execution of testing, a QA Engineer must manually define the test values to be used in the tests to be executed. Often times, these tests previously have been defined and the code and test descriptions for these tests have already been written. From practice it may be seen that these tests are often written with no re-usability of code in mind. Blocks of test code are often manually duplicated across the test suites. There is not a current mechanism that allows QA Engineers to define test values in a declarative manner, much less for the declarative combinatorial testing.
Combinatorial testing is a method that can reduce cost and increase the effectiveness of software testing for many applications. The key insight underlying this form of testing is that not every parameter contributes to every failure and most failures are caused by interactions between relatively few parameters. Combinatorial testing can be employed in all scenarios where it is necessary to verify that subject of testing is able to pass validation with more than one value of given variable, especially if there are several such variables. Current methods implemented in modern testing frameworks are often cumbersome.